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ABSTRACT 


The  basic  adaptive  filtering  algorithm  =  XE  -  eYn(YnXn  “  ^n) 

is  analyzed  ,  via  the  theory  of  weak  convergence.  Apart  from  some 
very  special  cases,  the  analysis  is  hard  when  done  for  each  fixed 
e  >  0.  But  the  weak  convergence  techniques  are  set  up  to  provide  much 
information  for  small  e.  The  relevant  facts  from  the  theory  are 
given.  Define  xE(-)  by  xE(t)  =  XE  on  [ne,n£  +  e).  Then  weak 
(distributional)  convergence  of  { xe ( - ) >  and  of  {xE(-  +  t  )}  is 


proved  under  very  weak  assumptions,  where  t£  -*•  00  as  e  -*•  0.  The 


normalized  errors 


Xn  -  0 


Ji 


are  analyzed,  where  0  is  a  'stable' 


point  for  the  'mean'  algorithm.  We  also  develop  the  asymptotic  prop¬ 
erties  of  a  projection  algorithm,  where  the  XE  are  truncated  at 


each  iteration,  if  they  fall  outside  of  a  given  set. 


1.  Introduction. 


-  )  - 


y  This  paper  illustrates  the  power  of  weak  convergence  methods 
through  the  analysis  of  the  basic  algorithm  of  adaptive  filtering. 


(1.1)  Xjj+1  =  X£  -  eYn{YnX£  -  ipn),  X£€R  ,  Euclidean  r-space. 


n'  n  n 


•fy  Sub  nj 


Except  for  the  simplest  cases  (e.g.,  when  (Y  }  ■  are  mutually  inde- 

fu  {  ^ 

pendent),  the  analysis  of  -fl .  V)  for  fixedc^  is  difficult.  However, 
asymptotic  analysis'^ e~-*~ Ojjvia  weak  convergence  methods  provides  much 
information,  relatively  ^inlessly.  Define  the  interpolated  process 


xe(-)  by 

(1.2) 


\ 


fitnit  0  f  O) 


xb(t)  =  X£  on  [nc,ne  +  e). 


Let  {tj  denote  a  sequence  which  goes  to  °°  as  e  -*•  0. 

In  the  next  section,  we  review  some  definitions  and  results  con¬ 
cerning  weak  convergence.  In  Section  3,  the  weak  convergence  of 
(x£(- )}  is  studied,  and  in  Section  4,  we  examine  the  limit  problem 
for  {xG(t£  +  •))  to  get  a  clearer  picture  of  the  asymptotic  behavior. 
In  Section  5,  the  normalized  errors  are  studied,  and  a  very  useful  pro¬ 
jection  or  truncation  algorithm  is  dealt  with  in  Section  7. 


Weak  Convergence.  Weak  convergence  is  an  extension  of  the  notion 
of  convergence  in  distribution  to  random  variables  with  values  in  a 
function  space.  The  sequence  (xe(-)}  of  (1.2)  can  be  viewed  as  a 
sequence  of  random  variables  with  paths  in  the  function  space  D  [0,»), 
the  space  of  Revalued  functions  on  [0,°°)  which  are  right  continuous 
and  have  left  hand  limits.  As  will  be  seen,  this  point  of  view  is  very 
useful  in  applications.  This  space  is  discussed  in  Billingsley  [1]  and 
Kurtz  [2],  two  excellent  references  for  weak  convergence  theory. 

Under  the  so  called  "Skorohod  topology"  ([1],  Section  14),  Dr[0,°°)  is 
separable  and  metrizable,  and  the  metric  is  complete. 

The  space  Dr[0,°°)  is  useful  for  two  main  reasons.  First,  pro¬ 
cesses  with  paths  in  Dr[0,°°)  arise  naturally  in  applications  (e.g., 
the  xe(-)  of  (1.2)).  Second,  its'  topology  is  weaker  than  that  of 
Cr[0,«),  the  space  of  Revalued  continuous  functions  on  [0,°°),  so 
that  the  criteria  for  compactness  are  less  stringent,  and  better  con¬ 
vergence  theorems  can  be  obtained,  even  if  the  paths  or  their  limits 
lie  in  Cr[0,°°). 

For  Revalued  random  variables  {  XnJ ,  we  say  that  { Xn }  converges 
weakly  (or  in  distribution)  to  X  iff 

(2.1)  Ef(Xn)  -  Ef(X) 

for  each  bounded,  real  valued  and  continuous  function  f(-).  If  Xn 
and  X  take  values  in  a  metric  space,  we  also  say  that  (Xnl  converges 


weakly  to  X  and  write  Xn  =>  X  if  (2.1)  holds.  When  the  random 
variables  Xn  are  functionis,  we  write  lower  case  xn(-).  Since 
f(x( * ))  =  (x(t^),  ...  ,  x(t^)}  is  a  continuous  function  from  Cr[0,°°) 
to  R  ,  weak  convergence  x  (;)  =>x(-)  in  C  [0,“)  implies  the  conver¬ 
gence  of  multivariate  distributions  {xn(ti),  i  <  £}  ^{xU^,  i  _<  £}. 
Similarly  on  Dr[0,“>),  if  the  limit  process  x(-)  is  continuous  w.p.l 
at  t] ,  ...  t j . 

For  Revalued  X  ,  the  Helly-Bray  Theorem  states  the  following: 

If  for  each  <5  >  0,  3  K.  compact  such  that  P{ X  €  K,}  >  1  -  6  for  all 

o  no  — 

n,  then  {Xn>  is  said  to  be  tight  and  it  has  a  subsequence  which  con¬ 
verges  in  distribution.  The  definition  of  tightness  carries  over  to 
metric  space  valued  { Xn) .  Prohorov's  Theorem  [1]  states  that  tightness 
of  { xn ( ■ ) >  implies  that  it  has  a  weakly  convergent  subsequence. 

In  the  sequel,  we  frequently  use  the  above  'subsequence'  result 
in  the  following  way.  First  tightness  is  proved.  Then  a  weakly  con¬ 
vergent  subsequence  is  extracted.  The  limit  of  this  subsequence  is 
then  characterized  as  the  solution  to  a  specific  ODE  (ordinary  differ¬ 
ential  equation)  or  SDE  (Ito  equation).  It  is  then  shown  that  the 
limit  process  x(-)  does  not  depend  on  the  subsequence.  Hence 
xn  =?>x(-). 

Let  (X£)  be  defined  by  X£+^  =  X£  +  eF£  n>  where  (F£  n,e  >0, 
n  <  <» }  is  uniformly  integrable,  and  define  x£(-)  as  in  (1.2).  Then 
(xc(*))  is  tight  in  Dr[0,°°)  and  all  (weak)  limit  processes  have 
continuous  paths.  The  assertion  follows  from  ([1],  Theorem  15.2). 
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Let  xn(-)  =>x(*)>  where  the  paths  are  in  Cr[ 0 ,<*>)  or  Dr[0,®). 
Since  we  are  concerned  with  weak  convergence  only,  the  probability 
space  is  not  important,  and  we  can  select  it  in  any  convenient  way, 
provided  only  that  the  distributions  of  each  xn(-)  and  of  x(-)  does 
not  change.  By  a  technique  known  as  Skorohod  imbedding  (13],  Theorem 
3.1.1)  one  can  choose  the  probability  space  such  that  xn(-)  -*■  x ( - )  w.p.l 
in  the  topology  of  the  (path)  space  Cr  or  Dr.  This  very  useful 
method  will  often  be  used  without  explicit  mention. 

The  Martingale  problem  .  In  this  paper,  all  limits  will  satisfy 
ODE's  or  SDE's.  From  the  point  of  view  of  usefulness  in  weak  conver¬ 
gence  analysis,  a  very  nice  way  of  characterizing  a  limit  process  x(-) 
is  to  show  that  it  satisfies  a  martingale  problem  (the  martingale  prob- 
lem  of  Stroock  and  Varadhan  [4]).  Let  Cq  denote  the  space  of  real 
valued  functions  on  Rr  with  compact  support  and  continuous  second 
partial  derivatives. 

Define  the  operator  y  on  Cq  by 

(2.2)  =  yx)b(x)  +  \  l  aiJ(x)fx  x  (x), 

1  ,j  i  j 

where  (a..(x)}  =  a(x)  =  cr(x)a'(x)  and  b(-)  and  a(’)  are  continuous. 

'  J 

Let  x(*)  be  a  random  process  with  paths  in  D  [0,»).  Define 


^(t)  =  f (x(t) )  -  |  i^f(x(u))du. 
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A  2 

If  M^(-)  is  a  martingale  for  each  f ( - ) €  Cq »  then  x(-)  is  contin¬ 
uous  and  is  said  to  satisfy  the  martingale  problem  for  operator  & . 
It  can  be  shown  that  there  is  a  Wiener  process  w(-)  such  that  x(-) 
satisfies  the  Ito  equation 

(2.3)  dx  =  b(x)dt  +  o(x)dw 


Let  x(-)  be  a  right  continuous  process  and  jy  a  countable  set. 
A  useful  method  to  show  that  x(-)  solves  the  martingale  problem  is 
to  show  that  for  arbitrary  f€  C^,  arbitrary  l,  and  arbitrary 
t-j<...<tJJ<t<t  +  s  not  in  07  and  arbitrary  h(-)  bounded 
and  continuous, 


(2.4) 


E[h(x(t1),  i  <  «,)[f(x(t  +  s))  -  f(x(t))  - 


t+s 

iz^f  (x(u )  )du]  ]=0 

n 


This  implies  that  x( • )  solves  the  martingale  problem  for  operator  y 

and  is  continuous.  The  x(-)  in  this  paper  will  usually  be  the  limit 

of  a  sequence  in  Dr[0,°°)  (e.g.,  of  the  xe(-)  introduced  in  Section 

1),  and  a-priori  we  do  not  know  that  it  is  continuous,  w.p.l.  If  it 

is  not  continuous  at  ti  or  t  or  t+s,  then  we  may  not  be  able 

to  show  that  (2.4)  holds.  But  there  are  at  most  countably  many  points  at  which 

x(-)€0r[0,»)  is  not  continuous  w.p.l  :  this  set  is  _9~.  Henceforth 

we  conveniently  ignore  . 

A  truncation  device.  In  order  to  prove  tightness,  it  is  sometimes 
useful  to  work  with  a  truncated  process.  Referring  to  (1.2),  for  each 
N,  let  x  ’  (•)  denote  a  process  which  equals  x  (•)  until  first 
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N 

exit  from  =  {x  :  |x|  <_  N}.  For  each  N,  let  there  be  x  (•)  such 

that  xe,N(-)  =>xN(-)»  where  x^(-)  solves  the  martingale  problem  for 
N 

an  operator^  defined  by 

i^Nf(x)  =  fx(x)bN(x)  +  j  J'  ai  j(x)fx.x .  (x) • 

1  *J  ^  J 

Let  bN(x)  =  b(x)  and  oN(x)  =  a(x)  for  xeS^,  and  let  the  martingale 
problem  for  operator  y  have  a  unique  solution.  Then  xG(-)  =>x(-)» 
the  solution  to  the  martingale  problem  for  operator  _/  [5,6]. 

Let  qN ( • )  denote  a  twice  continuously  differentiable  function 
which  equals  1  in  and  zero  on  Rr  -  .  For  the  system  (1.1), 

the  N-truncation  is  defined  by 


(2.5) 


w£,N  vC,N  y  /y'w£,N  V 

n+l  n  eWn  WXn  U 


e,N/.\  ve,N  r  .  ,  ve,N  ve 

x  ( t )  =  Xp’  or.  [ne,ne  +  e),  XQ’  =  XQ. 


Using  these  remarks, we  will  often  simply  proceed  in  the  analysis  as  if 
{x£(-)>  were  bounded. 

Notation.  When  no  confusion  arises,  we  write  t/e  for  the  larg¬ 
est  integer  i  such  that  i  <_  t/e  The  symbol  denotes  conditioning 
on  (X£,Yj,^,j  <  n}. 


^  PW  *>»• 
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3.  Weak  Convergence  of  {xe(  - )}. 


{Y  Y  ,Y  >  is  uniformly  integrable 
n  n  n  n  - — - - - 


Theorem  1 .  Let  Xq  =>Xq.  Assume  (3.1)  and  either  (3.2i)  or 
(3.2ii)  for  some  matrix  R  and  vector  B,  £S  n  -*•  °°  and  N  -+  °°. 

(3.1) 

(3. 2i ) 

.  imi  ,  _  ,  irm  p 

(3-211)  U„En V/*"'  *  B 


i  n+N  ,  p  ,  n+N  p 

H  .1  YjYj— R*  i  .1  Vj  —  B 

j=n  J  J  j=n  J  J 


Then  xe(-)  ^x(-),  which  satisfies 

(3.3)  x  =  -Rx  +  B,  x(0)  =  Xq. 


Remark.  Under  (3.1),  ( 3 . 2 i i )  is  implied  by  ( 3 . 2 i ) .  There  are 
several  approaches  to  obtaining  (3.3),  among  them  being  the  schemes  in 
[8].  The  method  here  has  the  advantage  of  being  easy  to  generalize. 
The  conditions  are  clearly  related  to  those  used  in  [6], [7], [8]. 
Reference  [7]  examines  a  recursive  algorithm  with  'state  dependent' 
noise. 

Proof.  We  work  with  the  N-truncation  and  show  x  ’  (•)  •+  x  (•) 

where 

(3.4)  =  (-RxN  +  B)qN(xN),  xN(0)  =  xQ. 
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As  noted  in  section  2,  this  implies  that  x£(*)  ■•x(-)  satisfying  (3.3). 

We  use  (3.2ii):  under  (3.2.i)  the  proof  is  similar.  Let  n£  satisfy 

n  -*■  »  and  cn  =  5  ->0  as  e  0.  Fix  N.  By  the  truncation  and 
tee 

(3.1),  {'|,n(YnXp’f'i  -  ipn  )q^  ( *  ^ )}  is  uniformly  integrate,  so  (xe,N(-)} 
is  tight  in  Dr[0,°°),  and  all  (weak)  limits  lie  in  Cr[0,°°).  Fix  a  conver- 
gent  subsequence  (also  indexed  by  e)  with  limit  x  (•)•  Fix  f  €  CQ 
and  define  g(X ,Y ,ip)  =  f^(X)-[-Y(Y  X  -  ip ) ] qN ( X ) .  Henceforth  we  suppress 
the  N  superscript  on  x£,f\t)  and  X?’^,  but  retain  it  for  the  limit 

=  j 

xN(0  .  Define  the  piecewise  constant  function  g£(-)  by 


Fix  l ,  t..  <  t  <  t  +  s,  i  <_  t,  and  let  h(-)  be  bounded  and  continuous. 
There  are  A£  such  that  | A?  [  -*■  0  as  e  -*•  0  and 


(t+s)/e 

(3.5)  Eh(x£(t . ) ,i  <  *)[f(x£(t  +  s))  -  f(x£(t))  -  e  £  g(X£  Y  .  .  )]«A? 

j=t/e  J  J  J 


Also,  by  the  properties  of  conditional  expectations,  since  t^  <  t, 

_  (t+s )/6 

(3.6)  Eh(x£(t.),i  <  £)[f(xe(t  +  s ))-f (x£(t) )-6  l  eg(MJl"A§, 

£  Zst/6  £  4 

£ 

ft+s 

(3.7)  Eh(x  (t.),i  <  Jl)[f(x£(t  +  s))  -  f(x£(t))  -  g£(u)du]  =  A~. 

1  it  6 


Below,  it  will  be  shown  that 

(3.8)  g£(v)  -*•  fx(xN(v)[-RxN(v)  +  B]qN(x^(v)),  each  v,  as  e  •+  0. 
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Assume  (3.8)  for  the  moment.  By  the  weak  convergence  and  Skorohod  im¬ 
bedding,  {xe(t.),  i  £  i,  xc(t),  xe(t  +  s)}  -»■  (xN(t.),  i  £  £,  xN(t), 

xN(t  +  s)}  w.p.l .  Using  this,  (3.8)  and  the  fact  that  sup  E|ge(v) |  <  ®  , 

e,V£t 

and  taking  limits  in  (3.7)  yields 

(3.9)  Eh(xN(ti),  i  £  £)[f(xN(t  +  s))  -  f(xN(t))  - 


|t+Sdu  f^(xN(u))[-RxN(u)  +  B]qN(xN(u))]  =  0. 


Eqn.  (3.9)  and  the  arbitrariness  of  f(-)»  h(*)»£>t^,t  and  t  +  s, 

N  N 

imply  that  x  (•)  solves  the  martingale  problem  for  the  operator  & 


defined  by 


^Nf(x)  =  f  (x)[-Rx  +  B]qN(x). 


Thus  (3.4)  holds,  and  we  need  only  prove  (3.8). 

Fix  v  and  let  16  -*■  v  as  e  0.  Define  m  =  l  n  .  By  the 

e  e  eee 

f  N 

weak  convergence  and  Skorohod  imbedding  x  (•)  ->  x  (•),  a  continuous 
£  N 

process.  Thus  X.  -*■  x  (v)  uniformly  for  je  U  n  ,£  n  +  n  )  as 

J  £  £  £  £  £ 

e  0.  Thus,  by  the  continuity  of  fx(')  and  dN(-)  and  (3.1), 

m  +n  -1 

£  £ 

lim  ge(v)  *  lim  -J-  l  E  f ’ (X?)C-Y -(Y*xf  -  ^-)]qN(xf) 
e-0  e*0  ne  j=m  mE  x  J  3  J  J  J  N  J 

t3jo> 

-  qM(xN(v))f>N(v)lim  i  V  -  *J)1. 
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where  all  limits  are  in  probability.  Finally,  by  (3.2ii)  and  the  weak 
convergence,  the  last  line  of  (3.10)  yields  (3.8). 

Q.E. 
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4.  Limits  as  e  -*•  0  and  en  «. 

Since  the  system  (1.1)  is  usually  in  operation  for  a  long  time, 
the  behavior  as  e  -*■  0  and  en  -*•  »  simultaneously  is  of  considerable 

interest.  Define  x£(-)  =  x£(t  +  •)•  where  t  -*■  «  as  e  0.  Weak 

e  e 

convergence  alone  does  not  imply  that  x£(-)  ■*  stationary  solution  to 

(3.3).  For  example,  define  y£(t)  =  max[0,  t  -  1/e].  Then  y£(-)  **  zero 

function,  but  lim  y£(t)  -*■  °°  for  each  e.  However,  under  slightly 
t 

stronger  conditions  than  used  in  the  last  section, we  do  get  the  desired 
limit  as  e  0,  t  -*■  °°.  First,  we  prove  Theorem  2,  and  then  a  criter¬ 
ion  for  the  tightness  will  be  given.  If  R  >  0,  define  0  =  R  ^B. 

Theorem  2.  Let  (Xf ,  e  >  0,  j  <  »}  be  tight  in  Rr,  and  assume 


sJ 

(3.1)  and  either  (3.2i  or  ii).  Then  {xe(-))  is  tight  and  all  weak 


limits  satisfy  (3.3).  If  R  >  0,  then  the  weak  limit  is  the  constant 


function  with  value  e,  the  stationary  solution. 

Proof  outline.  The  first  assertion  follows  from  the  proof  of 
Theorem  1.  We  need  only  characterize  the  limit  process  when  R  >  0. 

To  do  this,  we  exploit  the  stability  of  (3.3).  For  any  T  <  <*>,  take  a 
weakly  convergent  subsequence  of  the  pair  (xe(-)»  xe(-  -  T)},  with 
limit  (x(- ),xT(- )).  We  have  x(Q)  =  xT(T).  The  set  of  possible 
(Xj(0)}  is  tight  (over  all  T  and  convergent  subsequences),  since 
(X^,e  >  0,j  <  «°}  is.  Thus,  the  conclusion  is  implied  by  the  arbitrari- 
ness  of  T  and  the  representation 

xt(T)  =  x(0)  =  (exp  -  RT ) xT ( 0 )  +  f  (exp  -  R(T  -  t))Bdt  = 

1  J0 

(exp  -  RT)xj(O)  +  O  -  j^(exp  -  Rt)Bdt. 
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The  next  proof  Illustrates  the  use  of  the  'perturbed  Liapunov 
function*  method  16], [9], [10].  It  exploits  the  stability  of  (3.3)  to 
obtain  tightness  of  the  iterates  and  of  the  sequence  of  normalized 
errors. 


Theorem  3.  Let  R  >  0  and  let  be  bounded.  Suppose 


that 


l  lEnYjYj  -  Rl>  l  lEnVj  ‘  B! 

j=n  J  J  j=n  n  J  J 


is  bounded,  uniformly  in  n  and  let  {Xq}  be  tight.  Then  there  are 

N  <  «>  such  that 

e  - 

(4.1)  {X^,  e  small,  j  <  »}  is  tight 

J 

(4.2)  {(X?  -  e)//e,  e  small,  j  >  N„}  is  tight. 

J  £ 

(If  (Xs:  -  0)  =  0(/e),  we  can  set  N  =  0). 

J  £ 

Proof.  Recall  that  9  =  R”^B  and  define  $e  =  Xe  -  6.  Then 
-  n  n 

(4.3)  «;tI  -  *  *<Vn  *  Vn  9  >' 

Henceforth,  suppress  the  e  superscript  on  6^  and  suppose  without 

i 

loss  of  generality  that  {Xq}  is  uniformly  bounded.  Define  V(5)  =66 
and  define  the  function  We(n)  by 

E(n>  =  2e5h  -l  Wj  ■  R)5n  -  2c5n  X  MVj  - 
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The  K  below  are  real  numbers  which  do  not  depend  on  e  and  whose 
values  might  change  from  usage  to  usage.  Define  the  perturbed  Liapunov 
function  Ve(n)  =  V ( 6n )  -  We(n). 

We  have 


(4.4) 

|We(n)|  <  Kell  +  V(5n)] 

|We(n) |  <  KeCl  +  |Ve(n)|J. 

Also 

Env<W  -  V(«n)=-2e6>, 

~2eS ' E  ( Y  Y '  -  R)6  +  2e6* E  (Y  i|»  -  Y  /  0  ) 
n  n  n  n  n  n  n  nrn  n  n 

+  0(e2)(l  +  |<5j2), 

E„WE(n  +  1)  -  Ue(n)  =  -2^E„(Y„v;  -  R)«„ 

+  2^nE„(Yn*n-V>)+0^>IlSnl  +  l«nl2>- 

Thus  there  is  a  X  >  0  such  that  for  small  e  >  0, 

+  1)  -  Ve(n)  =  -2e<5^R6n  +  0(e2)[1  +  l6J2] 

<_  -eXVe{ n)  +  O(e^) , 

<  (1  -  eX)nEVe(0)  +  0(e) 

<  K(1  -  eX)nEV(<50)  +  0(e). 


E^(n 


which  implies  that 
EVe(n) 


(4.5) 


EV(6n) 


The  theorem  follows  from  (4.5). 


Q.E.D. 
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5.  Local  Behavior  Near  0 . 

Under  the  conditions  of  Theorem  3,  TTm  E|x!j  -  9|^  =  0(e).  In 

n  n 

fact,  we  can  do  much  better.  Define  =  (Xn  -  Q)/Si  .  Using  any 

N  satisfying  the  needs  of  Theorem  3,  define  ue ( * )  by  ue(0)  =  U^ 

e  e 

and  ue(t)  =  .  on  [je,je  +  e).  Let  t  -*■  «  as  e  -*■  0  and  set 

ue(t)  =  ue(t  +  t£).  Theorem  4  shows  that  {ue ( * ) >  converges  weakly  to 
a  certain  Gaussian  diffusion.  The  properties  of  this  diffusion  are 
quite  helpful  for  our  understanding  of  the  effects  of  the  noise  and 
stability  properties  on  the  algorithm. 

I 

Theorem  4.  Def i ne  £n  =  (Yn^n  -  Y^e),  and  let 

n+t/e 

(5.1)  Wf(t)  =  l  Sit,. 

j=n  J 

converge  weakly  to  a  (possibly  non-standard)  Wiener  process  w(-)  with 
covariance  £t,  a^  e  -*■  0  and^  n  °°.  Assume  (3 . 2i 

or  ii)  and  the  conditions  of  Theorem  3.  Then  is  tight  and  each 

e 

subsequence  of  {ue ( • ) }  (resp.,  {ue(’)>)  contains  a  further  subsequence 
which  converges  to  a  solution  to  (5.2)  (resp. ,  to  the  stationary  solution 
to  (5.2)).  I_f  {U§>  is  tight,  then  we  can  use  N£  =  0  and  the  asser¬ 
tion  concerning  convergence  of  {ue(*))  remains  valid  if  the  conditions 
of  Theorem  3  are  replaced  by  (3.1). 

(5.2)  du  =  -Rudt  +  Bdt  +  dw 
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Remark.  Assuming  the  weak  convergence  (5.1)  is  much  more  conven¬ 
ient  than  stating  conditions  which  guarantee  it,  since  such  convergence 
is  the  subject  of  a  large  literature.  See,  e.g.,  [1]. 

Proof.  The  proof  is  basically  a  modification  of  that  of  Theorem 
1 .  We  have 

<5'3>  Cl  ■  “n  -  eYnYnun  *  R  V 


The  tightness  of  (ujj}  follows  from  Theorem  3.  For  notational  simpli¬ 


city,  set  =  0.  To  prove  the  theorem  properly,  we  should  use  the 

N-truncation  u  ’  (•)  of  u  (•)  which  is  defined  by 

=  u^,N  -  £YnY;CNvCN) +  ^n‘  and  then  show  that 

ue,N(-)  •*  uN(-),  where  uN(-)  satisfies  duN  =  -RuNqN(uN)dt  +  Bdt  +  dw, 
and  then  let  N  -»■  «  to  obtain  (5.2).  But,  in  order  to  save  notation 
and  a  few  details,  we  ignore  the  truncation,  and  simply  suppose  that 

p  4* 

the  (u^)  are  bounded  .  By  this  boundedness  and  the  uniform  integra- 
bility  of  { YnYn , Yn^n >  and  the  convergence  Wp(-)  «*  w(-),  we  have 
t/e  .  _ 

that  {e  l  Y.Y.in}  and  (W^ ( • ) >  are  tight  and  that  all  limits  have 

Q  J  J  J  " 

continuous  paths.  Thus  (ue(-))  is  tight,  and  all  limits  have  con¬ 
tinuous  paths. 


+  £  N 

Actually,  the  }  are  not  bounded,  but  for  each  T  <  ®  we  have 

lim  Tim  P{  supjiflN|  >  K}  =  0.  This  and  the  cited  uniform  integrability 
K-**>  e  je<T  J  ~ 

and  convergence  of  W^(-)  to  w(*)  are  enough  to  get  tightness  of 
{ut,N(-)}»  with  all  limits  being  in  Cr[0,°°). 
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Define  the  functions  g£(t)  and  Ge(t)  by 
Jtn£+n  -1 

9£<t)'^  J  V'YM  +  Vj)-°nU6e'l6etStK 

Ge(t)  =  f  g£(u)du. 

>  0 

First,  we  work  with  {ue(  - ) } . 

We  have 


(5.4)  ue(t)  -  u£(0)  =  G£(t)  +  W£(t)  +  le  £  (-Y.Y.U!  +  Y.*.  )  -  G£(t)] 

U  J  J  J  J  J 


The  process  in  the  bracket  in  (5.4)  is  tight  and  converges  weakly 
to  the  zero  process,  as  t  -*  °°.  This  can  be  shown  by  a  slight  modifi¬ 
cation  of  the  method  of  Theorem  1.  The  sequence  {u£( • )  ,Wq( • )}  is 
tight.  Extract  a  weakly  convergent  subsequence  with  limit  (u(- ),w(*))» 

p  ft 

and  indexed  also  by  e.  By  the  method  of  Theorem  1,  G  (t)  •+  (-Ru(s)+B)ds 

i  0 

for  each  t. 

The  limits  of  the  different  convergent  subsequences  differ  only 
in  the  initial  condition  u(0).  The  argument  for  (u£(-))  is  the 
same  as  that  for  {u£(.))»  except  for  the  stationarity  part.  But  this 
part  follows  from  an  argument  like  that  used  in  Theorem  2.  The  last 
assertion  follows  from  the  previous  ones,  since  if  {U g }  is  tight, 
we  do  not  need  to  prove  the  existence  of  N£  <  °°,  and  so  the  bounded- 

I 

ness  of  { YpYn ,Ynipn>  can  be  replaced  by  (3.1)  and  a  truncation  argument, 
when  working  with  (u£(-)h 


Q.E.D. 
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6.  Tracking  Parameter  Variations. 

If  the  statistics  of  Y . change  with  time  and  their  "rate  of 
change’  is  commensurate  with  e,  then  Theorem  1  can  be  extended.  Let 
there  be  continuous  R(-)  and  B(-)  such  that  for  each  t,  (3.2) 
holds  if  ne  ->  t  and  R(t)  and  B(t),  replace  R  and  B,  resp.  Then, 
under  (3.1)  Theorem  1  continues  to  hold. 

7.  A  Projection  or  Truncation  Algorithm. 

In  practical  problems,  the  iterates  { X^}  are  usually  prevented 
from  becoming  too  large  by  using  a  projection  or  truncation,  and  we  now 
treat  a  simple  case.  Define  the  box  H  =  {x  :  | x . j  <  k,  i  <  r}.  Let 
tth(x)  denote  the  closest  point  in  H  to  x,  and  let  n(x,h)  denote 
the  projection  onto  H  defined  by 

n(x,h)  =  limOnfx  +  Ah)  -  x]/A. 

A-K)  H 

Thus  tt ( x , h )  =  h  if  x  is  interior  to  H,  or  if  h  points  ’inside1 
when  x€3H,  the  boundary  of  H.  We  treat  the  algorithm 

(M)  C,  ■  nH[<  -  *»„<VS  -  *„».  *o-  «o£H- 

Define  sx  =  x-  e*x-  R_1B,  and  write  F(x)  =  E(Y^x  -  *n)2.  If 
(Yn’^n^  1s  statl'°nary  and  R  >  0,  F(x)  is  strictly  convex.  The  con¬ 
straints  that  define  the  box  H  can  be  written  as  q,.(x)  <  0, 
i  a  1,  ...  ,2r,  where  the  q^(x)  are  of  the  form  x^  -  k  and 
-k  -  x.j  .  A  point  x€  H  is  said  to  be  a  Kuhn-Tucker  point  iff  there 


i 
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are  A.  >  0  such  that 

Fjx)  +  I  Aiqi  (x)  *  0, 

X  i€A(x)  1  1,X 

where  A{x)  denotes  the  set  of  constraints  which  are  active  at  x. 
In  the  present  case,  x  being  a  Kuhn-Tucker  point  is  necessary  and 
sufficient  for  its  minimizing  F(-)  on  H. 

Theorem  5.  Assume  (3.1)  and  either  (3.2i)  or  (3.2ii).  Then 
xe( • )  x(-).  where 


(7.2)  x  =  tt(x,  -Rx  +  B),  Xq  =  x(0), 
or,  equivalently, 

(7.3)  <5x  =  ir(x,  -R6x). 


Let  R  >  0  and  let  tYn ,^n>  be  stationary,  and  define  x£(-)  as  in 
Theorem  2,  then  xe(-)  ■»  stationary  solution  to  (7.3),  which  is  a  con¬ 
stant  solution  and  a  Kuhn-Tucker  point  for  the  problem  of  minimizing 
F(x)  subject  to  x€H. 

Proof.  Rewrite  (7.1)  in  the  form 


Xn+1  =  X^  "  eYn(YnXn  '  O  +  e2n» 

n+i  n  n  n  n  n'  n 


where 


<  -  VXn  -  ■  Ixn  •  '  *„» 


Thus  {z^,  e  >  0,  n  <  °°}  is  uniformly  integrable.  Using  the  notation  of 
Theorem  1,  define  the  functions  ze(*)  and  Ze(-)  by 
£n£+n  -1 

*E(t>  •  r  l  Vfx(Xj>2n  °"  t5Et5e>- 

X.I  £ 

ft 

Ze(t)  =  ze(u)du. 

Jo 

Define  ge(*)  and  Ge(-)  as  in  Theorem  1.  Since  {X^}  is  bounded  by 
the  projection,  no  N-truncation  is  needed.  By  (3.1),  (xe( • ) ,Ze( • )}  is 
tight  and  all  limits  are  continuous.  Also,  the  limits  of  (Ze(-))  are 
all  absolutely  continuous  because  of  the  uniform  integrability  of  { z^} . 
Choose  a  weakly  convergent  subsequence  indexed  by  e  and  with  limit 
(x(*),Z(*)).  Define  z(-)  by  Z(t)  =  J  z(u)du.  By  the  method  of 
Theorem  1 

ft  ft 

x(t)  =  x(0)  +  (-Rx(u)  +  B)du  +  z(u)du. 

JO  JO 

Note  that  z^  =  0  if  Xc.,  is  interior  to  H.  Also,  if 
n  n+ 1 

X^+i €  9H,  then  z^  is  a  non-negative  linear  combination  of  the  inward 
normals  to  the  'active  surfaces'  at  x  *  X^ .  This  implies  the  following 
for  almost  all  t.  If  x(t)  is  interior  to  H,  then  z(t)  =  0.  If 
x(t)€3H,  then  z(t)  is  a  non-negative  linear  combination  of  the  inward 
normals  to  the  'active  surfaces'  at  x  =  x(t).  From  this  and  the  geometry 
of  H,  we  conclude  that  x(-)  satisfies  (7.2)  or  (7.3). 

We  now  discuss  the  stability  of  (7.3).  Let  R  >  0  and  define 
V(x)  =  6xR6x.  Then  V(6x)  =  26x'Rtt(x,-R<5x).  If  v(-)  is  continuous, 
then  the  function  K(v(x))  =  v(x)'tt(x,v(x)) 


is  non-positive  and  upper  semi  continuous  in  v.  It  is  zero  only  at 
those  x  where  ir(x,v(x))  =  0.  Thus  6x(t)  converges  to  the 
point  6x  where  tt(x,-R6x)  =  0.  This  6x  is  a  Kuhn-Tucker  point 
since  either  (a):  x  is  interior  to  H,  in  which  case  5x  =  0  and 
R6x  =  0,  or  (b):  x€  3H,  in  which  case  R6x  must  be  a  non-negative 
linear  combination  of  the  inward  normals  of  the  constraints  which  are 
active  at  x.  From  this  point  on,  the  proof  of  the  convergence  of 
{xe(*)}  to  the  stationary  solution  (the  Kuhn-Tucker  point)  is  -essen¬ 
tially  the  same  as  the  convergence  proof  for  {xL(-)}  in  Theorem  2 
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